Device and method for transmitting video

ABSTRACT

Various embodiments of the disclosure relate to a video transmittance device and method capable of robust response to network bandwidth changes by changing the resolution or frame rate of a video based on a dual-bandwidth threshold which is a reference value for determining whether to change the resolution or frame rate of a video signal and a network bandwidth prediction value determined based on predetermined control information received from a video reception device. The video transmission device may include: receiving a video signal, setting a dual-bandwidth threshold which is a reference value for determining whether to change a resolution or a frame rate of the video signal, determining a network bandwidth prediction value based on specified control information received from a video reception device, determining whether to change the resolution or frame rate of the video based on the network bandwidth prediction value and the dual-bandwidth threshold, updating frame sequence information based on the resolution or frame rate being changed, setting an encoding parameter based on the changed resolution or frame rate, encoding the video signal based on the set encoding parameter, and transmitting the encoded video signal to the video reception device.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of International Application No.PCT/KR2022/012095 designating the United States, filed on Aug. 12, 2022,in the Korean Intellectual Property Receiving Office and claimingpriority to Korean Patent Application No. 10-2021-0155700, filed on Nov.12, 2021, in the Korean Intellectual Property Office, the disclosures ofwhich are incorporated by reference herein in their entireties.

BACKGROUND Field

The disclosure relates to a video transmission device and method capableof robust response to network bandwidth changes by changing theresolution or frame rate of a video based on a dual-bandwidth thresholdwhich is a reference value for determining whether to change theresolution or frame rate of the video signal and a network bandwidthprediction value determined based on predetermined control informationreceived from a video reception device.

Description of Related Art

A video transmission device may share a video by streaming the video toa video reception device through an IP-based network. The videotransmission device may encode a video and generate a streaming datapacket and transmit it to the video reception device. The videoreception device may receive the streaming data packet, decode thevideo, and then play the video. A series of streaming data packetstransmitted by the video transmission device is called a stream.

The adaptive streaming service refers to a scheme for adaptivelychanging the quality of the streaming service, such as the resolutionand frame rate of the video depending on changes in network environment,such as transmission bandwidth, and transmitting it.

In a chunk-based adaptive streaming service, as an example of anadaptive streaming service, the video transmission device switchesstreams in chunk units depending on a change in network environment in astate of storing the video encoded with their respective resolutions andframe rates in the form of chunks having a predetermined length. Thus,it may not immediately respond to network variability and thus cause adelay.

To address such issues, in an adaptive streaming service which mayprovide streaming service in units of frames made up of chunks, eachresolution or frame rate period is divided by its correspondingthreshold, and the video transmission device changes the resolution orframe rate depending on a change in network environment based on thethreshold and streams it. Thus, the frame-based adaptive streamingservice allows for low-latency streaming as compared with thechunk-based adaptive streaming service. However, in the frame-basedadaptive streaming service, the sensitivity to network bandwidthvariability is increased, so that the resolution or frame rate may befrequently changed in frame units and, as the resolution or frame rateis changed, the infra frames (I frames) which arenon-compressed/reference frames which require independent encodingaccording to a change in resolution or frame rate are overly used,causing degradation of compression performance and resultant videoquality degradation.

FIG. 1 conceptually illustrates a scenario in which a video transmissiondevice changes the resolution of a video in response to networkbandwidth variability in a conventional frame-based adaptive streamingservice. Referring to FIG. 1 , the video transmission device divides thevideo into three resolution sections of 4K, 2K, and HD based on twothresholds of BW_TH1 and BW_TH2 and provides the video streamingservice. If the network bandwidth is sequentially varied as denoted bythe solid line, the video transmission device may change the resolutionaround the BW_TH1 resolution seven times and use the I frame seven timesaccording to the changes in resolution to perform encoding. Since the Iframe does not use inter-frame compression, its compression efficiencyis several tens or several hundreds of times lower than the predictedframe (P frame) or bidirectional frame (B frame). Accordingly, theconventional frame-based adaptive streaming service may encounterdrastic degradation of compression performance and video quality due touse of many I frames within a short time.

In providing an adaptive streaming service, when network bandwidthvariability is sensitive due to frequent variations in network bandwidtharound the threshold corresponding to each resolution or frame ratesection, the video transmission device needs a robust response to thenetwork bandwidth variability by suppressing frequent variations inresolution or frame rate while adaptively streaming in real-time.

SUMMARY

Embodiments of the disclosure provide a video transmission device andmethod that changes the resolution or frame rate of a video based on adual-bandwidth threshold which is a reference value for determiningwhether to change the resolution or frame rate of the video signal, anda network bandwidth prediction value determined based on predeterminedcontrol information received from the video reception device.

According to an example embodiment of the disclosure, a videotransmission device may comprise: a communication unit comprisingcommunication circuitry configured to transmit and/or receive datathrough a network, a video input unit comprising circuitry configured toreceive a video signal, a memory storing a dual-bandwidth thresholdcomprising a reference value for determining whether to change aresolution or a frame rate of the video signal, a video pre-processingunit comprising circuitry configured to: set the dual-bandwidththreshold, store the dual-bandwidth threshold in the memory, determine anetwork bandwidth prediction value based on specified controlinformation received from a video reception device by controlling thecommunication unit, and determine whether to change the resolution orframe rate of the video signal based on the network bandwidth predictionvalue and the dual-bandwidth threshold, a video encoding unit comprisingcircuitry configured to: upload frame sequence information based on theresolution or frame rate being changed, set an encoding parameter basedon the changed resolution or frame rate, and encode the video signalbased on the set encoding parameter, and a video output unit comprisingcircuitry configured to control the communication unit to transmit theencoded video signal to the video reception device.

According to an example embodiment of the disclosure, a videotransmission method may comprise: receiving a video signal, setting adual-bandwidth threshold including a reference value for determiningwhether to change a resolution or a frame rate of the video signal,determining a network bandwidth prediction value based on specifiedcontrol information received from a video reception device, determiningwhether to change the resolution or frame rate of the video based on thenetwork bandwidth prediction value and the dual-bandwidth threshold,updating frame sequence information based on the resolution or framerate being changed, setting an encoding parameter based on the changedresolution or frame rate, encoding the video signal based on the setencoding parameter, and transmitting the encoded video signal to thevideo reception device.

According to various example embodiments of the disclosure, the videotransmission device may adaptively encode video considering the networkbandwidth variability and may thus transmit high-quality video at alow-latency.

When the network bandwidth is frequently varied around the thresholdcorresponding to each resolution or frame rate so that the networkbandwidth variability is sensitive, the video transmission device maydetermine whether to change the resolution or frame rate of the videobased on the dual-bandwidth threshold or multi-bandwidth threshold, thusreducing the frequency of changing the resolution or frame rate,preventing and/or reducing overuse of the non-compressed I frame, andhence preventing and/or reducing degradation of compression performanceand video quality, and allowing for robust response to network bandwidthvariability.

It is possible to provide an integrated viewing experience capable offreely viewing or sharing video seamlessly, anywhere in the house,through low-latency, high-quality video streaming.

Effects of the disclosure are not limited to the foregoing, and otherunmentioned effects would be apparent from the following description.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features, and advantages of certainembodiments of the present disclosure will be more apparent from thefollowing detailed description, taken in conjunction with theaccompanying drawings, in which:

FIG. 1 is a diagram illustrating a scenario in which a videotransmission device changes the resolution of a video in response tonetwork bandwidth variability in a conventional frame-based adaptivestreaming service;

FIG. 2 is a diagram illustrating an example system of providing anadaptive streaming service according to various embodiments;

FIG. 3 is a block diagram illustrating an example configuration of avideo transmission device providing an adaptive streaming serviceaccording to various embodiments;

FIGS. 4 and 5 are diagrams illustrating example scenarios in which avideo transmission device changes the resolution of a video in responseto network bandwidth variability according to various embodiments;

FIG. 6 is a graph illustrating a correlation between per-resolution bitrate and viewing quality according to various embodiments; and

FIG. 7 is a flowchart illustrating an example video transmission methodaccording to various embodiments.

DETAILED DESCRIPTION

For use in describing the various example embodiments of the disclosure,common terms widely used as possible have been chosen consideringfunctions in the disclosure, but the terms may be varied depending onthe intent of one of ordinary skill in the art or case laws or theadvent of new technologies. In certain cases, some terms may bearbitrarily selected, and in such case, their detailed definitions maybe given in the relevant parts thereof. Accordingly, the terms usedherein should be determined based on their meanings and the overalldisclosure, rather than by the terms themselves.

When an element “includes” another element, the element may furtherinclude the other element, rather excluding the other element, unlessparticularly stated otherwise. Further, the terms “unit,” “module,” or“part” as used herein denote a unit processing at least one function oroperation, and a unit, module, or part may be implemented in hardware,software, or a combination thereof.

Embodiments of the disclosure are now described with reference to theaccompanying drawings. However, the disclosure may be implemented inother various forms and is not limited to the embodiments set forthherein. In the drawings, the same reference numerals refer to the sameelements, and the size of each component in the drawings may beexaggerated for clarity of description.

FIG. 2 is a diagram illustrating an example system of providing anadaptive streaming service according to various embodiments.

Referring to FIG. 2 , an adaptive streaming service providing system 200according to an embodiment may include a video transmission device 210and a video reception device 220. The video transmission device 210 maychange the resolution or frame rate depending on a variation in networkenvironment, encode the video frame, and provide the video stream (orvideo signal) to the video reception device 220, and the video receptiondevice 220 may receive and play the video stream from the videotransmission device 210. The video transmission device 210 may determinea network bandwidth prediction value according to the variation innetwork environment based on predetermined control information receivedfrom the video reception device 220 or may receive the network bandwidthprediction value determined by the video reception device 220 from thevideo reception device 220. For example, the video transmission device210 may determine the network bandwidth prediction value according tothe variation in network environment considering the networktransmission time of the acknowledgment (ACK) received from the videoreception device 220, but it will be appreciated that other varioustypes or pieces of control information may be used. When the videoreception device 220 determines the network bandwidth prediction valueand transmits the network bandwidth prediction value to the videotransmission device 210, the video reception device 220 may determinethe network bandwidth prediction value based on previously transmittedvideo stream transmission information, but it will be appreciated thatthe network bandwidth prediction value may be determined by othervarious methods. The video transmission device 210 may determine theresolution or frame rate on a per-frame basis based on the networkbandwidth prediction value, thereby providing a low-latency videostreaming service adaptively considering the variation in networkenvironment. The video transmission device 210 may include various videosource devices, such as a TV, a PC, a smartphone, a tablet, a set-topbox, and a game console, and the video reception device 220 may includevarious video playback devices, such as a TV, a smartphone, a tablet,and a PC. It is apparent that the video transmission device 210 and thevideo reception device 220 are not limited to a specific type ofdevices.

The video transmission device 210 and the video reception device 220 maytransmit/receive a video stream and control information through thenetwork 230. According to various embodiments, the network 230connecting the video transmission device 210 and the video receptiondevice 220 may include a short-range communication network, such aswireless fidelity (Wi-Fi), or a remote communication network, such as acellular network, a next-generation communication network, the Internet,or a computer network (e.g., LAN or WAN), and may communicate based onan IP communication protocol. The cellular network may include globalsystem for mobile communications (GSM), enhanced data GSM environment(EDGE), code division multiple access (CDMA), time division multiplexingaccess (TDMA), 5G, long-term evolution (LTE), and LTE-advanced (LTE-A).The network 230 may include connections of network components, such ashubs, bridges, routers, switches, and gateways. The network 230 mayinclude one or more connected networks, e.g., a multi-networkenvironment, including public networks such as the Internet and privatenetworks such as safe business private networks. Access to the network230 may be provided via one or more wired or wireless access networks.Further, the network 230 may support Internet-of-things (IoT) networks,which exchange information between distributed components, e.g., things,and process the information.

FIG. 3 is a block diagram illustrating an example configuration of avideo transmission device providing an adaptive streaming serviceaccording to various embodiments.

Referring to FIG. 3 , the video transmission device 300 may include avideo input unit (e.g., including video input circuitry) 310, a videopre-processing unit (e.g., including processing circuitry) 320, a videoencoding unit (e.g., including video encoding circuitry) 330, and avideo output unit (e.g., including video output circuitry) 340. Thevideo transmission device 300 may further include a memory (not shown),a processor (not shown), and a communication unit (not shown). The videotransmission device 300 may include additional components in addition tothe illustrated components, or may omit at least one of the illustratedcomponents.

According to an embodiment, the memory may store a program including oneor more instructions or data such as configuration information. Thememory may be configured as a volatile memory, a non-volatile memory, ora combination of a volatile memory and a non-volatile memory. The memorymay provide stored data according to a request of the processor.

According to an embodiment, the communication unit may include variouscommunication circuitry and provide an interface for communication withother systems or devices. The communication unit may include a networkinterface card or a wireless transmission/reception unit that enablescommunication through an external network (e.g., a network). Thecommunication unit may perform signal processing for accessing awireless network. The wireless network may include, e.g., at least oneof a wireless LAN or a cellular network (e.g., long-term evolution(LTE)).

According to an embodiment, the processor may include various processingcircuitry, may be electrically connected to the communication unit andthe memory and may execute computation or data processing regardingcommunication and/or control of at least one other component of thevideo transmission device 300, using the program stored in the memory.The processor may execute at least one instruction corresponding to thevideo input unit 310, the video pre-processing unit 320, the videoencoding unit 330, and the video output unit 340. The processor mayinclude at least one of a central processing unit (CPU), a graphicprocessing unit (GPU), a micro controller unit (MCU), a sensor hub, asupplementary processor, a communication processor, an applicationprocessor, an application specific integrated circuit (ASIC), or fieldprogrammable gate arrays (FPGA) and may have multiple cores.

According to an embodiment, the video input unit 310 may include variousvideo input circuitry and receive a video frame. The video frame may bereceived from the outside of the video transmission device 300 or may beembedded in the video transmission device 300. The video input unit 310may control the communication unit to receive the video frame from theoutside in a wired or wireless scheme.

According to an embodiment, the video pre-processing unit 320 mayinclude various video processing circuitry and determine the networkbandwidth prediction value based on predetermined control informationreceived from the video reception device 220 or may obtain the networkbandwidth prediction value received from the video reception device 220.For example, the video pre-processing device 320 may determine thenetwork bandwidth prediction value according to the variation in networkenvironment considering the network transmission time of theacknowledgment (ACK) received from the video reception device 220, butit will be appreciated that other various types or pieces of controlinformation may be used. When the video reception device 220 determinesthe network bandwidth prediction value and transmits the networkbandwidth prediction value to the video transmission device 300, thevideo reception device 220 may determine the network bandwidthprediction value based on previously transmitted video streamtransmission information, but it will be appreciated that the networkbandwidth prediction value may be determined by other various methods.

The video pre-processing unit 320 may set a dual-bandwidth threshold andstore it in the memory, and may determine whether the resolution orframe rate of the video is changed based on the network bandwidthprediction value and the set dual-bandwidth threshold. Thedual-bandwidth threshold may include an upper bandwidth threshold, whichis a reference value for determining whether to increase the resolutionor frame rate from a first resolution or first frame rate to a secondresolution or second frame rate, or a lower bandwidth threshold, whichis a reference value for determining whether to decrease the resolutionor frame rate from the second resolution or second frame rate to thefirst resolution or first frame rate. The video pre-processing unit 320may set the dual-bandwidth threshold based on an external input.

If the network bandwidth prediction value exceeds the upper bandwidththreshold, the video pre-processing unit 320 may change the resolutionor frame rate from the first resolution or first frame rate to thesecond resolution or second frame rate. If the network bandwidthprediction value is less than the lower bandwidth threshold, the videopre-processing unit 320 may change the resolution or frame rate from thesecond resolution or second frame rate to the first resolution or firstframe rate.

According to an embodiment, the video encoding unit 330 may includevarious video encoding circuitry and update frame sequence informationwhen the resolution or frame rate is changed. The frame sequenceinformation may refer, for example, to header information containinginformation related to encoding or decoding of the frame sequence, suchas profile, level, resolution, or frame rate. For example, the framesequence information may include H.264 codec sequence parameter set(SPS), but it is apparent that it is not limited thereto. The profilemay refer, for example, to a collection of the minimum necessarytechnical elements and functions for a representative application systemamong the technical elements of the encoding/decoding operation of thevideo.

The video encoding unit 330 may set encoding parameters based on theresolution or frame rate determined by the video pre-processing unit320. The encoding parameter may include at least one piece of encodinginformation, such as frame rate, intra/inter mode prediction, in-loopfilter, or slice type, but it is apparent that it is not limitedthereto. Inter-mode prediction may refer, for example, to an encodingscheme for generating a prediction block from the picture coded at aprevious time. Intra-mode prediction may refer, for example, to anencoding scheme for generating a prediction value for the block to becoded from coded adjacent pixels in the current picture. The in-loopfilter may refer to a technique for enhancing the quality of thereconstructed video, and may include a deblocking filter, a sampleadaptive offset (SAO), and an adaptive loop filter (ALF). The deblockingfilter may remove block distortion occurring at the boundary betweenblocks in the reconstructed picture. The SAO may add an appropriateoffset value to the pixel value to compensate for a coding error. TheALF may perform filtering based on a value obtained by comparing thereconstructed video with the original video after the block is filteredthrough the deblocking filter. Slice type indicates the type of slice,which is the basic unit for encoding, and may include P, B, I, SP, SIslices, and the like. The P slice may refer, for example, to the slicewhere intra-picture encoding or inter-picture prediction coding usingone reference picture is performed. The B slice may refer, for example,to the slice where intra-picture encoding or inter-picture predictioncoding using two reference pictures is performed. The I slice may refer,for example, to the slice where only intra-picture encoding isperformed. The SP slice may refer, for example, to a special P slice forperforming bit string conversion. The SI slice may refer, for example,to a special I slice for performing bit string conversion. The videoencoding unit 330 may encode the first frame after the resolution orframe rate is changed, with the I slice independently from the previousframes. The video transmission device 300 may encode the frame based onthe set encoding parameter.

According to an embodiment, the video output unit 340 may includevarious video output circuitry and control the communication unit totransmit a video signal including the encoded frame through the network230 to the video reception device 220.

FIGS. 4 and 5 are diagrams illustrating an example scenario in which avideo transmission device changes the resolution of a video in responseto network bandwidth variability according to various embodiments.

Referring to FIG. 4 , the video pre-processing unit 320 may set a firstdual-bandwidth threshold including a lower bandwidth threshold BW_TH1_L,which is the reference value for determining whether to decrease theresolution of the video from 4K to 2K, and an upper bandwidth thresholdBW_TH1_H, which is the reference value for determining whether toincrease the resolution of the video from 2K to 4K, based on an externalinput. Further, the video pre-processing unit 320 may set another firstdual-bandwidth threshold including a lower bandwidth threshold BW_TH2_L,which is the reference value for determining whether to decrease theresolution of the video from 2K to HD, and an upper bandwidth thresholdBW_TH2_H, which is the reference value for determining whether toincrease the resolution of the video from HD to 2K, based on an externalinput.

Referring to the illustrated example, the video pre-processing unit 320may change the resolution of the video from the 2K resolution to the 4Kresolution when the network bandwidth prediction value exceeds the upperbandwidth threshold BW_TH1_H. The video pre-processing unit 320 maychange the resolution of the video from the 4K resolution to the 2Kresolution when the network bandwidth prediction value is less than thelower bandwidth threshold BW_TH1_L. In the illustrated example, thevideo pre-processing unit 320 may perform two resolution changes basedon the first dual-bandwidth threshold and the network bandwidthprediction value. Since the video encoding unit 330 encodes the firstframe after the resolution is changed, with the I frame independent fromthe previous frames, I frame encoding may be performed two times.

According to an embodiment, when the network bandwidth is frequentlyvaried around the threshold corresponding to each resolution or framerate so that the network bandwidth variability is sensitive, the videopre-processing unit 320 may determine whether to change the resolutionor frame rate of the video based on the first dual-bandwidth threshold,thus reducing the frequency of changing the resolution or frame rate,preventing and/or reducing overuse of the non-compressed I frame, andhence preventing and/or reducing degradation of compression performanceand video quality, and allowing for robust response to network bandwidthvariability.

FIG. 6 is a graph a correlation between per-resolution bit rate andviewing quality according to various embodiments.

Referring to FIG. 6 , a high-resolution video, such as of 4K, providesexcellent quality but, due to the increase in the number of pixels,undergoes an increase in compression rate as compared withlow-resolution video if the transmission bit rate of the network reducesand may thus be subject to severe quality degradation. Referring to theillustrated example, the viewing qualities of the 4K video and the 2Kvideo may cross at a bit rate of 7 Mbps, and the 2K video and the HDvideo may cross at a bit rate of 3 Mbps. The viewing quality may bemeasured by evaluation methods, such as mean opinion score (MOS) andvideo multimethod assessment fusion (VMAF), but it is apparent that itmay be measured using other various evaluation methods.

Accordingly, the video pre-processing unit 320 may reduce the frequencyof changing the resolution without degradation of the viewing qualityalthough setting a dual-bandwidth threshold including the lowerbandwidth threshold BW_TH1_L and the upper bandwidth threshold BW_TH1_Haround the bit rate where the viewing qualities cross, such as 7 Mbpsand determining whether the resolution of the video is changed based onthe dual-bandwidth threshold, thus allowing for a robust response tonetwork bandwidth variability.

Referring to FIG. 5 , the dual-bandwidth threshold may further include asecond dual-bandwidth threshold in addition to the first dual-bandwidththreshold described above with reference to FIG. 4 . The seconddual-bandwidth threshold may include a second upper bandwidth threshold(BW_TH1_HH, BW_TH2_HH), which is a reference value for determiningwhether to increase the resolution or frame rate from a first resolutionor first frame rate to a second resolution or second frame rate, and asecond lower bandwidth threshold (BW_TH1_LL, BW_TH2_LL), which is areference value for determining whether to decrease the resolution orframe rate from the second resolution or second frame rate to the firstresolution or first frame rate. The second upper bandwidth threshold maybe a predetermined value larger than the first upper bandwidththreshold, and the second lower bandwidth threshold may be apredetermined value smaller than the first lower bandwidth threshold.

If there is a resolution or frame rate change history within apredetermined time, the video pre-processing unit 320 may determinewhether to change the resolution or frame rate based on the networkbandwidth prediction value and the second dual-bandwidth threshold. Inother words, if there is a history of having the resolution or framerate within the predetermined time, the video pre-processing unit 320may determine whether to change the resolution or frame rate based onthe second dual-bandwidth threshold which has a larger interval betweenthe second upper bandwidth threshold and the second lower bandwidththreshold and thus robustly respond to network bandwidth variability.

Referring to FIG. 5 , in a case where if there is a resolution changewithin a time of previous five frames, if the network bandwidthprediction value exceeds the second upper bandwidth threshold BW_TH1_HH,the video pre-processing unit 320 may change the resolution of the videofrom the 2K resolution to the 4K resolution. Further, in a case where ifthere is a resolution change within a time of previous five frames, ifthe network bandwidth prediction value is less than the second lowerbandwidth threshold BW_TH1_LL, the video pre-processing unit 320 maychange the resolution of the video from the 4K resolution to the 2Kresolution. In the illustrated example, since there is a resolutionchange from 4K to 2K in the fifth frame, in the case of the eighthframe, it is determined whether the resolution is changed based on thesecond dual-bandwidth threshold and the network bandwidth predictionvalue. Thus, the video pre-processing unit 320 may determine not tochange the resolution.

According to an embodiment, the video pre-processing unit 320 may setthe second dual-bandwidth threshold based on an external input. Thevideo pre-processing unit 320 may dynamically set the seconddual-bandwidth threshold so that the difference between the second upperbandwidth threshold and the second lower bandwidth threshold has apredetermined value corresponding to the time difference between thevideo frame, where at least one of the resolution and frame rate hasbeen changed most recently, and the current video frame. In other words,the second dual-bandwidth threshold may be dynamically set so that asthe time difference from the frame where at least one of the resolutionand the frame rate has been changed most recently decreases, theinterval between the second upper bandwidth threshold and the secondlower bandwidth threshold is increased by a predetermined value.

According to an embodiment, if there is a change in resolution or framerate within the predetermined time, the video pre-processing unit 320may determine not to change the resolution or frame rate.

According to an embodiment, the video pre-processing unit 320 may setmulti-bandwidth threshold including at least one dual-bandwidththreshold respectively corresponding to at least one network bandwidthvariation level. The network bandwidth variation level may be determinedusing statistical values, such as the mean band value and variance,based on previous network bandwidth information during a predeterminedtime, but it is apparent that it may be determined by other variousmethods. The multi-bandwidth threshold may be set in such a manner as toincrease the interval between the upper bandwidth threshold and lowerbandwidth threshold as the network bandwidth variation level increaseswhile decreasing the interval between the upper bandwidth threshold andlower bandwidth threshold as the network bandwidth variation leveldecreases. The multi-bandwidth threshold may be dynamically set usingthe network bandwidth variation level as a variable. The videopre-processing unit 320 may determine whether to change the resolutionor frame rate of the video based on the dual-bandwidth thresholdcorresponding to the variation level of the network bandwidth predictionvalue during a predetermined time, of the multi-bandwidth threshold.

FIG. 7 is a flowchart illustrating an example video transmission methodaccording to various embodiments.

According to an embodiment, the video transmission device 300 mayreceive a video frame (or video signal). The video frame may be receivedfrom the outside of the video transmission device 300 or may be embeddedin the video transmission device 300.

According to an embodiment, the video transmission device 300 maydetermine the network bandwidth prediction value based on predeterminedcontrol information received from the video reception device 220 or mayobtain the network bandwidth prediction value received from the videoreception device 220.

According to an embodiment, the video transmission device 300 may set adual-bandwidth threshold which is a reference value for determiningwhether to change the resolution or frame rate of the video signal andstore the dual-bandwidth threshold in the memory.

According to an embodiment, in operation 710, the video transmissiondevice 300 may determine whether to change the resolution or frame rateof the video based on the network bandwidth prediction value and thedual-bandwidth threshold.

According to an embodiment, the dual-bandwidth threshold may include afirst dual-bandwidth threshold. The first dual-bandwidth threshold mayinclude a first upper bandwidth threshold, which is a reference valuefor determining whether to increase the resolution or frame rate from afirst resolution or first frame rate to a second resolution or secondframe rate, and a first lower bandwidth threshold, which is a referencevalue for determining whether to decrease the resolution or frame ratefrom the second resolution or second frame rate to the first resolutionor first frame rate. The video transmission device 300 may previouslyset the first dual-bandwidth threshold based on an external input. Ifthe network bandwidth prediction value exceeds the first upper bandwidththreshold, the video transmission device 300 may change the resolutionor frame rate from the first resolution or first frame rate to thesecond resolution or second frame rate. If the network bandwidthprediction value is less than the first lower bandwidth threshold, thevideo transmission device 300 may change the resolution or frame ratefrom the second resolution or second frame rate to the first resolutionor first frame rate.

According to an embodiment, the dual-bandwidth threshold may include asecond dual-bandwidth threshold. The second dual-bandwidth threshold mayinclude a second upper bandwidth threshold, which is a reference valuefor determining whether to increase the resolution or frame rate fromthe first resolution or first frame rate to the second resolution orsecond frame rate, and a second lower bandwidth threshold, which is areference value for determining whether to decrease the resolution orframe rate from the second resolution or second frame rate to the firstresolution or first frame rate. The second upper bandwidth threshold maybe a predetermined value larger than the first upper bandwidththreshold, and the second lower bandwidth threshold may be apredetermined value smaller than the first lower bandwidth threshold. Ifthere is a resolution or frame rate change history within apredetermined time, the video transmission device 300 may determinewhether to change the resolution or frame rate based on the networkbandwidth prediction value, the second upper bandwidth threshold, andthe second lower bandwidth threshold. The video transmission device 300may previously set the second dual-bandwidth threshold based on anexternal input. Alternatively, the video transmission device 300 maydynamically set the second dual-bandwidth threshold so that thedifference between the second upper bandwidth threshold and the secondlower bandwidth threshold has a predetermined value corresponding to thetime difference between the video frame, where at least one of theresolution and frame rate has been changed most recently, and thecurrent video frame.

According to an embodiment, if there is a change in resolution or framerate within the predetermined time, the video transmission device 300may determine not to change the resolution or frame rate.

According to an embodiment, the video transmission device 300 may setmulti-bandwidth threshold including at least one dual-bandwidththreshold respectively corresponding to at least one network bandwidthvariation level. In this case, the video transmission device 300 maydetermine whether to change the resolution or frame rate of the videobased on the dual-bandwidth threshold corresponding to the variationlevel of the network bandwidth prediction value during a predeterminedtime, of the multi-bandwidth threshold.

The video transmission device 300 may perform operation 720 when theresolution or frame rate is changed or may perform operation 740 whenthe resolution or frame rate is not changed.

In operation 720, the video transmission device 300 may change theresolution or frame rate of the video based on the network bandwidthprediction value and the dual-bandwidth threshold.

In operation 730, the video transmission device 300 may update framesequence information. The frame sequence information may refer, forexample, to header information containing information related toencoding or decoding of the frame sequence, such as profile, level,resolution, or frame rate.

In operation 740, the video transmission device 300 may set an encodingparameter based on the determined resolution or frame rate.

In operation 750, the video transmission device 300 may encode a videoframe (or a video signal) based on the determined resolution or framerate. The video transmission device 300 may transmit the encoded videosignal to the video reception device 220.

The method executed by the video transmission device 300 and the videoreception device 220 may be implemented as a hardware component, asoftware component, and/or a combination thereof. The method may beexecuted by any system capable of executing computer readableinstructions.

Software, as used herein, may include computer programs, codes,instructions, or combinations of one or more thereof and may configurethe processing device as it is operated as desired or may instruct theprocessing device independently or collectively. The software may beimplemented as a computer program including instructions stored incomputer-readable storage media. The computer-readable recording mediamay include, e.g., magnetic storage media (e.g., read-only memory (ROM),random-access memory (RAM), floppy disk, hard disk, etc.) and anoptically readable media (e.g., CD-ROM or digital versatile disc (DVD).Further, the computer-readable recording media may be distributed tocomputer systems connected via a network, and computer-readable codesmay be stored and executed in a distributed manner. The media may beread by a computer, stored in a memory, and executed by a processor.

The computer-readable storage medium may be provided in the form of anon-transitory storage medium. Here, the “non-transitory” storage mediummay not include a signal and is tangible, but this term does notdifferentiate between where data is semi-permanently stored in thestorage medium and where data is temporarily stored in the storagemedium.

According to embodiments of the disclosure, the program may be includedand provided in a computer program product. The computer programproducts may be traded as commodities between sellers and buyers. Thecomputer program products may include software programs orcomputer-readable storage media storing the software programs. Forexample, the computer program products may include software program-typeproducts (e.g., downloadable applications (apps)) that areelectronically distributed through manufacturers of electronic devicesor electronic markets (e.g., Google Play Store or App Store). Forelectronic distribution, at least part of the software programs may bestored in storage media or temporarily generated. In this case, thestorage media may be storage media of relay servers that temporarilystore the software programs, servers of electronic markets, or serversof manufacturers.

The computer program product may include a storage medium of a server ora storage medium of a device in a system including the server and thedevice. Alternatively, when there is a third device (e.g., a smartphone) that is communicatively connected to the server or device, thecomputer program product may include a storage medium of the thirddevice. Alternatively, the computer program product may include thesoftware program itself transmitted from the server to the device or thethird device, or transmitted from the third device to the device. Inthis case, one of the server, the device, and the third device mayexecute the computer program product to perform the method according tothe disclosed embodiments. Alternatively, two or more of the server, thedevice, and the third device may execute the computer program product todistributively implement the method according to the disclosedembodiments. For example, the server may execute the computer programproduct stored in the server to control the device communicativelyconnected with the server to perform the method according to thedisclosed embodiments. As another example, the third device may executethe computer program product to control the device communicativelyconnected to the third device to perform the method according to thedisclosed embodiment. When the third device executes the computerprogram product, the third device may download the computer programproduct from the server and execute the downloaded computer programproduct. Alternatively, the third device may execute the computerprogram product provided in a pre-loaded state to perform the methodaccording to the disclosed embodiments.

While the disclosure has been illustrated and described with referenceto various example embodiments, it will be understood that the variousexample embodiments are intended to be illustrative, not limiting. Itwill be further understood by one of ordinary skill in the art thatvarious changes or modifications may be made without departing from thetrue spirit and full scope of the disclosure including the appendedclaims and their equivalents. It will also be understood that any of theembodiment(s) described herein may be used in conjunction with any otherembodiment(s) described herein.

What is claimed is:
 1. A video transmission device, comprising: acommunication unit comprising communication circuitry configured totransmit and/or receive data through a network; a video input unitcomprising circuitry configured to receive a video signal; a memorystoring a dual-bandwidth threshold comprising a reference value fordetermining whether to change a resolution or a frame rate of the videosignal; a video pre-processing unit comprising circuitry configured to:set the dual-bandwidth threshold, store the dual-bandwidth threshold inthe memory, determine a network bandwidth prediction value based onspecified control information received from a video reception device bycontrolling the communication unit, and determine whether to change theresolution or frame rate of the video signal based on the networkbandwidth prediction value and the dual-bandwidth threshold; a videoencoding unit comprising circuitry configured to: update frame sequenceinformation based on the resolution or frame rate being changed, set anencoding parameter based on the changed resolution or frame rate, andencode the video signal based on the set encoding parameter; and a videooutput unit comprising circuitry configured to control the communicationunit to transmit the encoded video signal to the video reception device.2. The video transmission device of claim 1, wherein the dual-bandwidththreshold includes a first dual-bandwidth threshold, wherein the firstdual-bandwidth threshold includes a first upper bandwidth threshold,comprising a reference value for determining whether to increase theresolution or frame rate from a first resolution or first frame rate toa second resolution or second frame rate, and a first lower bandwidththreshold comprising a reference value for determining whether todecrease the resolution or frame rate from the second resolution orsecond frame rate to the first resolution or first frame rate, andwherein the video pre-processing unit is configured to set the firstdual-bandwidth threshold based on an external input.
 3. The videotransmission device of claim 2, wherein based on the network bandwidthprediction value exceeding the first upper bandwidth threshold, thevideo pre-processing unit is configured to change the resolution orframe rate from the first resolution or first frame rate to the secondresolution or second frame rate and, based on the network bandwidthprediction value being less than the first lower bandwidth threshold,the video pre-processing unit is configured to change the resolution orframe rate from the second resolution or second frame rate to the firstresolution or first frame rate.
 4. The video transmission device ofclaim 2, wherein the dual-bandwidth threshold further includes a seconddual-bandwidth threshold, wherein the second dual-bandwidth thresholdincludes a second upper bandwidth threshold, comprising a referencevalue for determining whether to increase the resolution or frame ratefrom the first resolution or first frame rate to the second resolutionor second frame rate and a second lower bandwidth threshold, comprisinga reference value for determining whether to decrease the resolution orframe rate from the second resolution or second frame rate to the firstresolution or first frame rate, wherein the second upper bandwidththreshold is a specified value larger than the first upper bandwidththreshold, and the second lower bandwidth threshold is a specified valuesmaller than the first lower bandwidth threshold, and wherein based onthere being a resolution or frame rate change history within a specifiedtime, the video pre-processing unit is configured to determine whetherto change the resolution or frame rate based on the network bandwidthprediction value and the second dual-bandwidth threshold.
 5. The videotransmission device of claim 4, wherein the video pre-processing unit isconfigured to set the second dual-bandwidth threshold based on anexternal input or set the second dual-bandwidth threshold so that adifference between the second upper bandwidth threshold and the secondlower bandwidth threshold has a specified value corresponding to a timedifference between a video frame, where at least one of the resolutionand frame rate has been changed most recently, and a current videoframe.
 6. The video transmission device of claim 1, wherein the videopre-processing unit is configured to determine not to change theresolution or frame rate based on there being a change in the frame rateor resolution within a specified time.
 7. The video transmission deviceof claim 1, wherein the video pre-processing unit is configured to: seta multi-bandwidth threshold including at least one dual-bandwidththreshold respectively corresponding to at least one network bandwidthvariation level, and determine whether to change the resolution or framerate of the video based on a dual-bandwidth threshold corresponding to avariation level of the network bandwidth prediction value during aspecified time, of the multi-bandwidth threshold.
 8. A videotransmission method, comprising: receiving a video signal; setting adual-bandwidth threshold comprising a reference value for determiningwhether to change a resolution or a frame rate of the video signal;determining a network bandwidth prediction value based on specifiedcontrol information received from a video reception device; determiningwhether to change the resolution or frame rate of the video based on thenetwork bandwidth prediction value and the dual-bandwidth threshold;updating frame sequence information based on the resolution or framerate being changed; setting an encoding parameter based on the changedresolution or frame rate; encoding the video signal based on the setencoding parameter; and transmitting the encoded video signal to thevideo reception device.
 9. The video transmission method of claim 8,wherein the dual-bandwidth threshold includes a first dual-bandwidththreshold, wherein the first dual-bandwidth threshold includes a firstupper bandwidth threshold, comprising a reference value for determiningwhether to increase the resolution or frame rate from a first resolutionor first frame rate to a second resolution or second frame rate, and afirst lower bandwidth threshold, comprising a reference value fordetermining whether to decrease the resolution or frame rate from thesecond resolution or second frame rate to the first resolution or firstframe rate, and wherein setting the dual-bandwidth threshold includessetting the first dual-bandwidth threshold based on an external input.10. The video transmission method of claim 9, further comprising: basedon the network bandwidth prediction value exceeding the first upperbandwidth threshold, changing the resolution or frame rate from thefirst resolution or first frame rate to the second resolution or secondframe rate; and based on the network bandwidth prediction value beingless than the first lower bandwidth threshold, changing the resolutionor frame rate from the second resolution or second frame rate to thefirst resolution or first frame rate.
 11. The video transmission methodof claim 9, wherein the dual-bandwidth threshold further includes asecond dual-bandwidth threshold, wherein the second dual-bandwidththreshold includes a second upper bandwidth threshold, comprising areference value for determining whether to increase the resolution orframe rate from the first resolution or first frame rate to the secondresolution or second frame rate and a second lower bandwidth threshold,comprising a reference value for determining whether to decrease theresolution or frame rate from the second resolution or second frame rateto the first resolution or first frame rate, wherein the second upperbandwidth threshold is a specified value larger than the first upperbandwidth threshold, and the second lower bandwidth threshold is aspecified value smaller than the first lower bandwidth threshold, andwherein determining whether to change the resolution or frame rate ofthe video includes, based on there being a resolution or frame ratechange history within a specified time, determining whether to changethe resolution or frame rate based on the network bandwidth predictionvalue and the second dual-bandwidth threshold.
 12. The videotransmission method of claim 11, wherein setting the dual-bandwidththreshold includes: setting the second dual-bandwidth threshold based onan external input; or setting the second dual-bandwidth threshold sothat a difference between the second upper bandwidth threshold and thesecond lower bandwidth threshold has a specified value corresponding toa time difference between a video frame, where at least one of theresolution and frame rate has been changed most recently, and a currentvideo frame.
 13. The video transmission method of claim 8, whereindetermining whether to change the resolution or frame rate of the videoincludes determining not to change the resolution or frame rate based onthere being a change in the frame rate or resolution within a specifiedtime.
 14. The video transmission method of claim 8, further comprising:setting a multi-bandwidth threshold including at least onedual-bandwidth threshold respectively corresponding to at least onenetwork bandwidth variation level; and determining whether to change theresolution or frame rate of the video based on the network bandwidthprediction value and a dual-bandwidth threshold corresponding to avariation level of the network bandwidth prediction value during aspecified time, of the multi-bandwidth threshold.